Preserve interception for offline Docker agents by xeophon · Pull Request #1841 · PrimeIntellect-ai/verifiers

xeophon · 2026-06-23T14:04:34Z

Overview

Preserves the framework routes required by an offline Docker harness while denying arbitrary agent egress. Trusted setup retains ordinary network access; after setup, the runtime moves to an isolated execution network for the remainder of its lifetime.

Details

Adds a runtime network-policy boundary that receives named framework routes and returns the URLs reachable from the sandbox. This keeps model and MCP routing independent from the Docker implementation and gives future sandbox backends one integration point for allowlists and blocklists.
Separates runtime placement from both runtime-to-host and host-to-runtime reachability, so tunnel and service exposure decisions describe the actual network topology.
Uses a per-runtime setup network, an isolated internal execution network, and one hardened nginx egress container. The agent loses the setup network when execution begins.
Gives every required route its own relay port and limits it to the declared URL path, including descendants. Direct host access, DNS, and arbitrary Internet egress remain unavailable, while container loopback still works.
Keeps the restriction active until the runtime is destroyed, so agent child processes cannot regain setup networking after the main harness process exits.
Supports model routes plus task-scoped, shared, and external MCP routes. Colocated services are rejected when a backend policy cannot preserve their required network direction.
Keeps the allow/block policy model internal until target syntax and backend enforcement are ready to become a supported public configuration surface.

Note

Preserve interception for offline Docker agents by routing egress through a restricted network

Adds network_access: bool to DockerConfig (docker.py); when False, the agent container is isolated from arbitrary egress after setup and all framework routes are proxied through a local nginx egress container on an isolated execution network.
Introduces docker_interception_host() to resolve the Linux bridge gateway IP so the interception server can bind an extra host address reachable from within the restricted container.
Extends InterceptionServer, StaticInterceptionPool, and ElasticInterceptionPool (server.py, pool.py) to accept an extra_host parameter and bind on multiple local addresses simultaneously when not tunneling.
Replaces runtime_is_local with runtime_reaches_host_locally throughout MCP launch and interception logic to correctly distinguish runtimes that need tunnels from those that reach host services directly.
Adds apply_network_policy to the Runtime base class (base.py); Rollout.run calls it after resolving endpoints so the harness runtime can rewrite model and MCP URLs before execution.
Risk: containers running with network_access=False use different networks during setup vs. execution; cleanup now removes the egress container and both networks, which may leave resources behind on hard failures.

^{Macroscope summarized 4d23b5a.}

Note

High Risk
Changes security-critical Docker network topology, URL rewriting for all harness traffic, and interception binding; misconfiguration could break rollouts or weaken isolation.

Overview
Introduces restricted agent networking for Docker harnesses via DockerConfig.network_access (default true). When false, setup keeps a normal bridge network; after rollout setup, apply_network_policy moves the agent onto an internal execution network and rewrites model/MCP URLs to per-route nginx egress proxies (egress:8080+N) that only allow the declared paths—arbitrary internet/DNS egress is blocked until teardown.

Adds a shared NetworkPolicy / Runtime.apply_network_policy hook on the base runtime (other backends still error if restricted). Rollout.run passes named routes (model, mcp:*) through this boundary and rejects colocated MCP/user servers when the harness policy cannot support them.

Refactors locality into explicit reachability: runtime_reaches_host_locally, reaches_host_locally, reachable_from_host_locally, and SharedToolServer.reaches_host (replacing local / harness_is_local in tunnel and MCP URL logic). Interception can listen on an extra_host (Docker bridge gateway from docker_interception_host) so containers without host loopback still hit /state and model routes when no public tunnel is used.

Tool/user runtimes with a restricted policy cannot be launched standalone (serve raises); expose failures now surface as SandboxError.

^{Reviewed by Cursor Bugbot for commit 4d23b5a. Bugbot is set up for automated code reviews on this repo. Configure here.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07e5e51513

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

macroscopeapp · 2026-06-23T14:10:52Z

Approvability

Verdict: Needs human review

This PR introduces significant new Docker networking infrastructure with network policies and egress control. Three unresolved review comments identify high-severity bugs: incorrect bridge gateway resolution, timing issues with interception host computation, and an attribute rename breaking shared tools functionality.

^{You can customize Macroscope's approvability policy. Learn more.}

# Conflicts: # verifiers/v1/ARCHITECTURE.md # verifiers/v1/GUIDE.md # verifiers/v1/interception/pool.py # verifiers/v1/interception/server.py # verifiers/v1/rollout.py # verifiers/v1/runtimes/__init__.py # verifiers/v1/runtimes/base.py # verifiers/v1/runtimes/docker.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1ac94fc1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# Conflicts: # verifiers/v1/runtimes/docker.py

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 81fed07. Configure here.}

cursor · 2026-07-14T17:43:05Z

+            "could not resolve the Docker host gateway: "
+            f"{(result.stderr or result.stdout).strip()}"
+        )
+    return gateway


Wrong Docker bridge gateway

High Severity

With network_access=False, the harness joins a per-runtime setup bridge, but docker_interception_host() reads the default bridge network gateway. That IP usually differs from the setup network gateway, so host interception and egress host.docker.internal mapping can point at the wrong address and framework routes fail from the container.

Additional Locations (2)

verifiers/v1/runtimes/docker.py#L157-L159

verifiers/v1/runtimes/docker.py#L324-L326

^{Reviewed by Cursor Bugbot for commit 81fed07. Configure here.}

cursor · 2026-07-14T17:43:05Z

+                and not runtime.network_access
+                and not tunneled
+                else None
+            )


Shared interception host too early

High Severity

Eval serving() fixes extra_host from docker_interception_host() before any rollout creates its setup network, and every rollout shares that single bind address. Per-rollout bridge gateways differ, so shared interception often never listens on the address restricted Docker harnesses use to reach the host.

Additional Locations (1)

verifiers/v1/interception/server.py#L175-L185

^{Reviewed by Cursor Bugbot for commit 81fed07. Configure here.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 81fed07b1a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-14T17:45:04Z


    url: str
-    local: bool
+    reaches_host: bool


Preserve the shared-tool locality attribute

Renaming this field leaves serve_tools() still reading server.local when attaching non-external shared tool servers to a rollout, so any Taskset.tools shared server launched by the framework now fails before the harness starts with AttributeError: 'SharedToolServer' object has no attribute 'local'. Update the caller to use reaches_host (or keep a compatible property) so shared tools continue to work.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4d23b5acf4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-14T18:13:40Z

+        if url is None:
+            raise SandboxError(f"{service.type} runtime cannot expose port {port}")
+        yield url
+    elif consumer_reaches_host:  # host network, direct consumer → localhost, no tunnel


Avoid Prime tunnels for local MCP routes

When the harness is DockerConfig(network_access=False) and a task-scoped or shared MCP tool uses the default non-colocated subprocess runtime, serve() sets consumer_reaches_host to false, so this branch exposes the otherwise local host tool via PrimeTunnel and the Docker sidecar whitelists that public URL. Even after the interception route is kept local, Docker-only offline evals with default host tools still fail on hosts without prime_tunnel credentials or external egress; local framework tool servers need a Docker-bridge/local URL path instead of falling through to the public tunnel branch.

Useful? React with 👍 / 👎.

xeophon · 2026-07-18T16:42:34Z

superseded by #2024

Add interception-only Docker networking

07e5e51

xeophon marked this pull request as draft June 23, 2026 14:07

Remove runtime tests from PR

7416022

chatgpt-codex-connector Bot reviewed Jun 23, 2026

View reviewed changes

Comment thread verifiers/v1/runtimes/docker.py Outdated

Use offline flag for Docker interception

b63ed87

xeophon changed the title ~~Add interception-only Docker networking~~ Preserve interception for offline Docker agents Jun 25, 2026

macroscopeapp Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread verifiers/v1/runtimes/docker.py Outdated

Use host interception port for offline Docker

83c11a6

xeophon marked this pull request as ready for review June 25, 2026 17:35

cursor Bot reviewed Jul 14, 2026

View reviewed changes

Comment thread verifiers/v1/rollout.py Outdated

Comment thread verifiers/v1/runtimes/docker.py Outdated

chatgpt-codex-connector Bot reviewed Jul 14, 2026

View reviewed changes

Comment thread verifiers/v1/runtimes/docker.py Outdated

xeophon added 3 commits July 14, 2026 13:01

refactor(v1): generalize runtime network policies

8d2399a

fix(v1): restore direct Docker interception

407b979

Merge remote-tracking branch 'origin/main' into docker-interception-only

81fed07

# Conflicts: # verifiers/v1/runtimes/docker.py

cursor Bot reviewed Jul 14, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 14, 2026

View reviewed changes

fix(v1): preserve shared tool reachability

4d23b5a

chatgpt-codex-connector Bot reviewed Jul 14, 2026

View reviewed changes

xeophon closed this Jul 18, 2026

Uh oh!

Conversation

xeophon commented Jun 23, 2026 • edited by macroscopeapp Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Details

Preserve interception for offline Docker agents by routing egress through a restricted network

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

macroscopeapp Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 14, 2026

Choose a reason for hiding this comment

Wrong Docker bridge gateway

Uh oh!

cursor Bot Jul 14, 2026

Choose a reason for hiding this comment

Shared interception host too early

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

xeophon commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

xeophon commented Jun 23, 2026 •

edited by macroscopeapp Bot

Loading

macroscopeapp Bot commented Jun 23, 2026 •

edited

Loading